The invention relates generally to a method and apparatus for transmitting sequences of images through a communications channel, and in particular, to a method and apparatus for providing improved motion compensation to the image sequence transmission process.
The transmission of sequences of images, and in particular sequences of naturally occurring images such as those represented by a television signal, has been the subject of a significant amount of investigation. Typically, investigators have relied upon the highly redundant nature of successive images in the sequence and have often modeled the image data as a three-dimensional Markov process with a correlation coefficient close to unity. The Markov model provides a motivation for utilizing differential pulse-code-modulation (DPCM) and transform coding techniques to take account of the interframe redundancy.
By analyzing the nature of typical moving video, it is easy to become convinced that the principal change occurring between successive frames is the inhomogeneous motion of the objects within the frame. It has also been recognized that an accurate apparatus and method of estimating and compensating for this spatially dependent motion enables the construction of an interframe data compression method and apparatus which can have substantially better performance than can be achieved by sending a signal representative merely of the difference between successive frames.
As a result, various motion compensating coding methods and apparatus have been developed. These systems typically are either receiver-based motion compensation systems or transmitter-based motion compensation systems. In the receiver-based motion compensation system, the receiver makes a prediction as to the motion and compensates the previous frame for the expected motion. The transmitter, operating in the same manner, then sends only an error signal describing what must be done at the receiver in order to correct the receiver predicted frame. The error signal is typically coded in order to reduce its bandwidth.
For a transmitter-based motion compensation system, the motion estimation process occurs only at the transmitter. Displacement vectors are generally determined over various regions of the image and this data is then transmitted to the receiver along with an error information data signal. At the receiver the compensation process is performed on the previously coded image first using the motion information provided by the transmitter. The error signal data provided by the transmitter is then added to the thus compensated receiver image in order to maintain picture quality.
Unfortunately, however, both of these systems are flawed in the manner in which they are implemented. One disadvantage of receiver-based motion compensation systems is that they are, by their very nature, predictive coding systems. Because the prediction is based upon picture elements which have already been transmitted and hence received, the receiver-based motion compensation coder does not have the ability to deterministically optimize the estimate which they make of the "displacement," i.e., the motion. Thus, a receiver-based motion compensation system using a traditional raster scanning procedure, which receives a sequence wherein a object is moving rapidly toward an upper left-hand portion of the image, cannot always properly compensate therefor. Thus, when the previous frame and the local picture elements in the present frame are found to have no displacement, and the prediction method will assume no displacement for the next picture element in the raster. However, since the next picture element has in fact been displaced many picture elements toward the upper left-hand portion of the image, a very large prediction error results. A transmitter-based motion compensation system, with knowledge of the entire image, would have avoided the error by deterministically selecting the displacement estimate which minimizes the actual error.
Another major disadvantage of receiver-based motion compensation systems is that they typically employ a differential pulse-code-modulation (DPCM) representation of the error signal. A traditional DPCM system employs a coding limit of one bit per picture element, since data is transmitted for each picture element. This limitation is avoided in motion compensation systems by first segmenting the image into its moving and non-moving parts. Then, entropy coded address information is transmitted to inform the receiver of the specific regions over which an error signal will be sent. In non-moving regions, no error information need be sent. Unfortunately, however, the amount of addressing information for such a DPCM system can often exceed an allowable, reasonable limit.
The major disadvantage of transmitter-based motion compensation systems which provide good image reproduction is that they require substantial overhead data to describe the motion vector field which is typically sent in PCM form. This problem is exacerbated when it is desired to have a high resolution representation of the vector field.
Many motion estimation methods associate with each pixel of the image a motion displacement vector which describes the movement of that pixel from one image to another. And some motion estimation methods provide, in the spatial-domain, information regarding the movement of a block or region of pixels from one frame to the next. When these methods are employed in connection with transmitter-based motion estimation systems, it is necessary to create at the transmitter a replica of the image which will be produced at the receiver.
The process of creating the estimated receiver image at the transmitter makes use of the motion estimation signal output either directly, or after coding. During this process, however, both the large amount of data provided when each pixel of the image has associated with it a motion vector, and the relatively coarse motion estimation definition provided for blocks or regions of an image, (with concomitant lower data requirements) have proved unsatisfactory. On the one hand, providing motion estimation data for each pixel is computationally inefficient and results in significant noise artifacts. The provision of the coarser motion estimation process has resulted in inaccuracies and imprecision in the resulting received images.
Accordingly, it is an object of the present invention to provide a motion compensation method and apparatus having high resolution without incurring the disadvantages of noise induced artifacts and large computational load. Other objects of the invention are an improved motion compensation method and apparatus which provide high image quality, ease of computation, and ease of implementation.